Papers with day-to-day scenarios
FanOutQA: A Multi-Hop, Multi-Document Question Answering Benchmark for Large Language Models (2024.acl-short)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models focus on intradocument dependencies or dependencies between a small number of documents. |
| Approach: | They propose to use a dataset of fan-out question-answer pairs and human-annotated decompositions with English Wikipedia as the knowledge base to evaluate models' reasoning. |
| Outcome: | The proposed dataset shows that models still have room to improve reasoning over inter-document dependencies in a long context. |